This script is the data visualization script that: 1) Makes QAQC file of all Secchi data from a digitized google sheet 2) Checks to make sure there are no duplicate in the file 3) Makes sure the maintenance log is working 4) generates figures to visualize both this past year and all combined years of data 5) Save the final EDI file 6) Downloads the qaqc function and the maintenance log for publication

REVIEWERS- If you are reviewing this data package replace the pasta link with the one from EDI. If there are questions ask the data point person.

Check for duplicates and gaps in the data frame

This section identifies if there are any duplicates. If there are duplicates. Look to see if they are true duplicates and then check the qaqc function to see how they were missed.

Are there any duplicates?

Reservoir Site DateTime Secchi_m

Flag Frequency

Let’s look at the flag Frequency for each variable. As a reminder here are the flag codes

Flag values for DateTime

0: no flag

1: Time set to 12:00:00 because an exact time was not recorded

Flag values for Secchi_m

0: no flag

1: value missing or removed and set to NA

2: multiple observations at a site at the same time are averaged together

3: value changed through the maintenance log

## [1] "Flag_DateTime"
## Flag_DateTime
##    0    1 <NA> 
##  573  181    0 
## [1] "Flag_Secchi_m"
## Flag_Secchi_m
##    0    2 <NA> 
##  750    4    0

Check to make sure that what is in the maintenance log was actually removed

Look at the last rows of the maintenance log

We want to make sure that our maintenance log actually worked and took out the values or changes those it was supposed to

Reservoir Site DataStream TIMESTAMP_start TIMESTAMP_end start_parameter end_parameter flag update_value notes
BVR 50 Secchi 2014-04-30 11:50:00 2014-04-30 12:10:00 Secchi_m NA 2 NA Averaged observations from the same site at the same time
BVR 50 Secchi 2014-07-23 11:50:00 2014-07-23 12:10:00 Secchi_m NA 2 NA Averaged observations from the same site at the same time
FCR 50 Secchi 2014-10-04 11:50:00 2014-10-04 12:10:00 Secchi_m NA 2 NA Averaged observations from the same site at the same time
CCR 50 Secchi 2021-08-19 12:15:00 2021-08-19 12:20:00 Secchi_m NA 2 NA Averaged observations from the same site at the same time

Check the that the columns have flags

Look at the first few rows of the data frame and check that the observations after the TIMESTAMP_start are flagged

Look at the first 5 rows for that time

DateTime Secchi_m Flag_Secchi_m
2021-08-19 12:18:00 2.35 2

Look at the last 6 rows for the maintenance time

Make sure the observations are flagged

QAQC Plots

## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_summary()`).
## Warning: Removed 31 rows containing missing values or values outside the scale range
## (`geom_point()`).

## Warning: Removed 26 rows containing missing values or values outside the scale range
## (`geom_point()`).

# convert DateTime to character
current_df$DateTime <- as.character(format(current_df$DateTime)) 
    

write_csv(current_df, 'secchi_depth_2013_2023.csv')

#list.files()